{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Garimpagem de Dados\n",
    "\n",
    "## Aula 4 - Exercídio de Classificação com kNN\n",
    "\n",
    "13/10/2017"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Dataset:** Titanic: Machine Learning from Disaster\n",
    "\n",
    "https://www.kaggle.com/c/titanic/data\n",
    "\n",
    "Partindo da aula passada:\n",
    "\n",
    "1. Atualizar a função que mede a distância euclidiana para o pacote do scikit-learn \n",
    "\n",
    "2. Implementar uma função que selecione os k vizinhos mais próximos (k > 1)\n",
    "\n",
    "3. Implementar uma função que recebe os k vizinhos mais próximos e determinar a classe correta\n",
    "\n",
    "4. Transformar as features categoricas em numéricas (tip: pandas ou scikit-learn)\n",
    "\n",
    "5. Analisar a necessidade de normalizar as features numéricas (tip: pandas ou scikit-learn)\n",
    "\n",
    "6. Selecionar as features baseada na correlação (tip: pandas)\n",
    "\n",
    "7. Separar o dataset em treino (75%) / teste (25%) / validação (10% do treino)\n",
    "\n",
    "4. Execute o classificador para 30 k's pulando de 4 em 4 e apresente todas as acurácias utilizando o dataset de validação (Qual o melhor k?) [plotar um gráfico com os resultados]\n",
    "\n",
    "5. Executar o classificador para o melhor k encontrado utilizando o dataset de teste e apresentar um relatório da precisão (tip: scikit-learn) [plotar um gráfico com os resultados]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn import datasets\n",
    "from sklearn.neighbors import DistanceMetric\n",
    "from collections import Counter\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn import metrics\n",
    "from sklearn.neighbors import KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class KNNClassifier(object):\n",
    "    def __init__(self):\n",
    "        self.X_train = None\n",
    "        self.y_train = None\n",
    "\n",
    "   # def euc_distance(self, a, b):\n",
    "   #     return np.linalg.norm(a-b)\n",
    "\n",
    "    def euc_distance(self, a, b):\n",
    "        dist = DistanceMetric.get_metric('euclidean')\n",
    "        ndarray = dist.pairwise([a, b])\n",
    "        distance = ndarray[0][-1]\n",
    "        return distance\n",
    "    \n",
    "    def closest(self, row):\n",
    "        distance_array = []\n",
    "        for i in self.X_train:\n",
    "            distance_array.append(self.euc_distance(i, row))\n",
    "            \n",
    "        nearest_neighbor = distance_array.index(min(distance_array))\n",
    "        return self.y_train[nearest_neighbor]\n",
    "\n",
    "    def k_closests(self, row, n_of_neighbors):\n",
    "        distance_array = []\n",
    "        k_nearest_array = []\n",
    "        for i in self.X_train:\n",
    "            distance_array.append(self.euc_distance(i, row))\n",
    "            \n",
    "        for i in range(0, n_of_neighbors):\n",
    "            k_nearest_array.append(distance_array.index(min(distance_array)))\n",
    "            del distance_array[distance_array.index(min(distance_array))]\n",
    "            \n",
    "        return self.y_train[k_nearest_array]\n",
    "    \n",
    "    def get_neighbor_class(self, neighbors):\n",
    "        return self.y_train[neighbors]\n",
    "    \n",
    "    def get_closest_class(self, neighbor_classes):\n",
    "        counter = Counter(neighbor_classes)\n",
    "        return counter.most_common(1)[0][0]\n",
    "    \n",
    "    def fit(self, training_data, training_labels):\n",
    "        self.X_train = training_data\n",
    "        self.y_train = training_labels\n",
    "\n",
    "    def predict(self, to_classify):\n",
    "        predictions = []\n",
    "        for row in to_classify:\n",
    "            label = self.closest(row)\n",
    "            predictions.append(label)\n",
    "        return predictions\n",
    "    \n",
    "    def predict_2(self, to_classify, n_of_neighbors):\n",
    "        predictions = []\n",
    "        for row in to_classify:\n",
    "            nearest_classes = self.k_closests(row, n_of_neighbors)\n",
    "            label = self.get_closest_class(nearest_classes)\n",
    "            predictions.append(label)\n",
    "        return predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Utilizando o dataset titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass  \\\n",
       "0            1         0       3   \n",
       "1            2         1       1   \n",
       "2            3         1       3   \n",
       "3            4         1       1   \n",
       "4            5         0       3   \n",
       "\n",
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  \n",
       "0      0         A/5 21171   7.2500   NaN        S  \n",
       "1      0          PC 17599  71.2833   C85        C  \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      0            113803  53.1000  C123        S  \n",
       "4      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('train.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Agora vamos remover colunas irrelevantes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.drop(['Ticket'], axis=1,inplace=True)\n",
    "df.drop(['Name'], axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass     Sex   Age  SibSp  Parch     Fare Cabin  \\\n",
       "0            1         0       3    male  22.0      1      0   7.2500   NaN   \n",
       "1            2         1       1  female  38.0      1      0  71.2833   C85   \n",
       "2            3         1       3  female  26.0      0      0   7.9250   NaN   \n",
       "3            4         1       1  female  35.0      1      0  53.1000  C123   \n",
       "4            5         0       3    male  35.0      0      0   8.0500   NaN   \n",
       "\n",
       "  Embarked  \n",
       "0        S  \n",
       "1        C  \n",
       "2        S  \n",
       "3        S  \n",
       "4        S  "
      ]
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Analisando os dados, percebemos que algumas linhas não possuem e informação da idade dos passageiros, vamos assumir, para essas linhas, a idade média dos passageiros"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"Age\"] = df.Age.fillna(df.Age.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### também há linhas com dados faltantes em cabine, então vamos remover estas linhas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.dropna(axis=0, how='any')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>38.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>35.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>54.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>51.8625</td>\n",
       "      <td>E46</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>16.7000</td>\n",
       "      <td>G6</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>58.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.5500</td>\n",
       "      <td>C103</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>male</td>\n",
       "      <td>34.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>D56</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>24</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>35.5000</td>\n",
       "      <td>A6</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>263.0000</td>\n",
       "      <td>C23 C25 C27</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>32</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>146.5208</td>\n",
       "      <td>B78</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>53</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>49.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>76.7292</td>\n",
       "      <td>D33</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54</th>\n",
       "      <td>55</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>65.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>61.9792</td>\n",
       "      <td>B30</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55</th>\n",
       "      <td>56</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>35.5000</td>\n",
       "      <td>C52</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62</th>\n",
       "      <td>63</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>83.4750</td>\n",
       "      <td>C83</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>66</th>\n",
       "      <td>67</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>female</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>F33</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>76</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>25.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.6500</td>\n",
       "      <td>F G73</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>89</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>263.0000</td>\n",
       "      <td>C23 C25 C27</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>93</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>46.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>61.1750</td>\n",
       "      <td>E31</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>97</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>71.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>34.6542</td>\n",
       "      <td>A5</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>98</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>63.3583</td>\n",
       "      <td>D10 D12</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>103</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>77.2875</td>\n",
       "      <td>D26</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110</th>\n",
       "      <td>111</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>52.0000</td>\n",
       "      <td>C110</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>118</th>\n",
       "      <td>119</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>247.5208</td>\n",
       "      <td>B58 B60</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>123</th>\n",
       "      <td>124</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>female</td>\n",
       "      <td>32.500000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>E101</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>125</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>54.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>77.2875</td>\n",
       "      <td>D26</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>128</th>\n",
       "      <td>129</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>22.3583</td>\n",
       "      <td>F E69</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>136</th>\n",
       "      <td>137</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>26.2833</td>\n",
       "      <td>D47</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137</th>\n",
       "      <td>138</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>37.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>139</th>\n",
       "      <td>140</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>79.2000</td>\n",
       "      <td>B86</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>149</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>male</td>\n",
       "      <td>36.500000</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>F2</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>151</th>\n",
       "      <td>152</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>66.6000</td>\n",
       "      <td>C2</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>748</th>\n",
       "      <td>749</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>D30</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>751</th>\n",
       "      <td>752</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>12.4750</td>\n",
       "      <td>E121</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>759</th>\n",
       "      <td>760</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>86.5000</td>\n",
       "      <td>B77</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>763</th>\n",
       "      <td>764</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>36.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>120.0000</td>\n",
       "      <td>B96 B98</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>765</th>\n",
       "      <td>766</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>77.9583</td>\n",
       "      <td>D11</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>772</th>\n",
       "      <td>773</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>female</td>\n",
       "      <td>57.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10.5000</td>\n",
       "      <td>E77</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>776</th>\n",
       "      <td>777</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>male</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.7500</td>\n",
       "      <td>F38</td>\n",
       "      <td>Q</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>779</th>\n",
       "      <td>780</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>211.3375</td>\n",
       "      <td>B3</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>781</th>\n",
       "      <td>782</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>17.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>57.0000</td>\n",
       "      <td>B20</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>782</th>\n",
       "      <td>783</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>D6</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>789</th>\n",
       "      <td>790</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>46.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>79.2000</td>\n",
       "      <td>B82 B84</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>796</th>\n",
       "      <td>797</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>49.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25.9292</td>\n",
       "      <td>D17</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>802</th>\n",
       "      <td>803</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>120.0000</td>\n",
       "      <td>B96 B98</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>806</th>\n",
       "      <td>807</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>A36</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>809</th>\n",
       "      <td>810</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>E8</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>815</th>\n",
       "      <td>816</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>B102</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>820</th>\n",
       "      <td>821</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>52.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>93.5000</td>\n",
       "      <td>B69</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>823</th>\n",
       "      <td>824</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>female</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>12.4750</td>\n",
       "      <td>E121</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>835</th>\n",
       "      <td>836</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>83.1583</td>\n",
       "      <td>E49</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>839</th>\n",
       "      <td>840</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>29.7000</td>\n",
       "      <td>C47</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>849</th>\n",
       "      <td>850</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>29.699118</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>89.1042</td>\n",
       "      <td>C92</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>853</th>\n",
       "      <td>854</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>16.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>39.4000</td>\n",
       "      <td>D28</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>857</th>\n",
       "      <td>858</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.5500</td>\n",
       "      <td>E17</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>862</th>\n",
       "      <td>863</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>48.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>25.9292</td>\n",
       "      <td>D17</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>867</th>\n",
       "      <td>868</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>31.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50.4958</td>\n",
       "      <td>A24</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>871</th>\n",
       "      <td>872</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>52.5542</td>\n",
       "      <td>D35</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>872</th>\n",
       "      <td>873</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0000</td>\n",
       "      <td>B51 B53 B55</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>879</th>\n",
       "      <td>880</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>83.1583</td>\n",
       "      <td>C50</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>887</th>\n",
       "      <td>888</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>female</td>\n",
       "      <td>19.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>B42</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>889</th>\n",
       "      <td>890</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>male</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>C148</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>202 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     PassengerId  Survived  Pclass     Sex        Age  SibSp  Parch      Fare  \\\n",
       "1              2         1       1  female  38.000000      1      0   71.2833   \n",
       "3              4         1       1  female  35.000000      1      0   53.1000   \n",
       "6              7         0       1    male  54.000000      0      0   51.8625   \n",
       "10            11         1       3  female   4.000000      1      1   16.7000   \n",
       "11            12         1       1  female  58.000000      0      0   26.5500   \n",
       "21            22         1       2    male  34.000000      0      0   13.0000   \n",
       "23            24         1       1    male  28.000000      0      0   35.5000   \n",
       "27            28         0       1    male  19.000000      3      2  263.0000   \n",
       "31            32         1       1  female  29.699118      1      0  146.5208   \n",
       "52            53         1       1  female  49.000000      1      0   76.7292   \n",
       "54            55         0       1    male  65.000000      0      1   61.9792   \n",
       "55            56         1       1    male  29.699118      0      0   35.5000   \n",
       "62            63         0       1    male  45.000000      1      0   83.4750   \n",
       "66            67         1       2  female  29.000000      0      0   10.5000   \n",
       "75            76         0       3    male  25.000000      0      0    7.6500   \n",
       "88            89         1       1  female  23.000000      3      2  263.0000   \n",
       "92            93         0       1    male  46.000000      1      0   61.1750   \n",
       "96            97         0       1    male  71.000000      0      0   34.6542   \n",
       "97            98         1       1    male  23.000000      0      1   63.3583   \n",
       "102          103         0       1    male  21.000000      0      1   77.2875   \n",
       "110          111         0       1    male  47.000000      0      0   52.0000   \n",
       "118          119         0       1    male  24.000000      0      1  247.5208   \n",
       "123          124         1       2  female  32.500000      0      0   13.0000   \n",
       "124          125         0       1    male  54.000000      0      1   77.2875   \n",
       "128          129         1       3  female  29.699118      1      1   22.3583   \n",
       "136          137         1       1  female  19.000000      0      2   26.2833   \n",
       "137          138         0       1    male  37.000000      1      0   53.1000   \n",
       "139          140         0       1    male  24.000000      0      0   79.2000   \n",
       "148          149         0       2    male  36.500000      0      2   26.0000   \n",
       "151          152         1       1  female  22.000000      1      0   66.6000   \n",
       "..           ...       ...     ...     ...        ...    ...    ...       ...   \n",
       "748          749         0       1    male  19.000000      1      0   53.1000   \n",
       "751          752         1       3    male   6.000000      0      1   12.4750   \n",
       "759          760         1       1  female  33.000000      0      0   86.5000   \n",
       "763          764         1       1  female  36.000000      1      2  120.0000   \n",
       "765          766         1       1  female  51.000000      1      0   77.9583   \n",
       "772          773         0       2  female  57.000000      0      0   10.5000   \n",
       "776          777         0       3    male  29.699118      0      0    7.7500   \n",
       "779          780         1       1  female  43.000000      0      1  211.3375   \n",
       "781          782         1       1  female  17.000000      1      0   57.0000   \n",
       "782          783         0       1    male  29.000000      0      0   30.0000   \n",
       "789          790         0       1    male  46.000000      0      0   79.2000   \n",
       "796          797         1       1  female  49.000000      0      0   25.9292   \n",
       "802          803         1       1    male  11.000000      1      2  120.0000   \n",
       "806          807         0       1    male  39.000000      0      0    0.0000   \n",
       "809          810         1       1  female  33.000000      1      0   53.1000   \n",
       "815          816         0       1    male  29.699118      0      0    0.0000   \n",
       "820          821         1       1  female  52.000000      1      1   93.5000   \n",
       "823          824         1       3  female  27.000000      0      1   12.4750   \n",
       "835          836         1       1  female  39.000000      1      1   83.1583   \n",
       "839          840         1       1    male  29.699118      0      0   29.7000   \n",
       "849          850         1       1  female  29.699118      1      0   89.1042   \n",
       "853          854         1       1  female  16.000000      0      1   39.4000   \n",
       "857          858         1       1    male  51.000000      0      0   26.5500   \n",
       "862          863         1       1  female  48.000000      0      0   25.9292   \n",
       "867          868         0       1    male  31.000000      0      0   50.4958   \n",
       "871          872         1       1  female  47.000000      1      1   52.5542   \n",
       "872          873         0       1    male  33.000000      0      0    5.0000   \n",
       "879          880         1       1  female  56.000000      0      1   83.1583   \n",
       "887          888         1       1  female  19.000000      0      0   30.0000   \n",
       "889          890         1       1    male  26.000000      0      0   30.0000   \n",
       "\n",
       "           Cabin Embarked  \n",
       "1            C85        C  \n",
       "3           C123        S  \n",
       "6            E46        S  \n",
       "10            G6        S  \n",
       "11          C103        S  \n",
       "21           D56        S  \n",
       "23            A6        S  \n",
       "27   C23 C25 C27        S  \n",
       "31           B78        C  \n",
       "52           D33        C  \n",
       "54           B30        C  \n",
       "55           C52        S  \n",
       "62           C83        S  \n",
       "66           F33        S  \n",
       "75         F G73        S  \n",
       "88   C23 C25 C27        S  \n",
       "92           E31        S  \n",
       "96            A5        C  \n",
       "97       D10 D12        C  \n",
       "102          D26        S  \n",
       "110         C110        S  \n",
       "118      B58 B60        C  \n",
       "123         E101        S  \n",
       "124          D26        S  \n",
       "128        F E69        C  \n",
       "136          D47        S  \n",
       "137         C123        S  \n",
       "139          B86        C  \n",
       "148           F2        S  \n",
       "151           C2        S  \n",
       "..           ...      ...  \n",
       "748          D30        S  \n",
       "751         E121        S  \n",
       "759          B77        S  \n",
       "763      B96 B98        S  \n",
       "765          D11        S  \n",
       "772          E77        S  \n",
       "776          F38        Q  \n",
       "779           B3        S  \n",
       "781          B20        S  \n",
       "782           D6        S  \n",
       "789      B82 B84        C  \n",
       "796          D17        S  \n",
       "802      B96 B98        S  \n",
       "806          A36        S  \n",
       "809           E8        S  \n",
       "815         B102        S  \n",
       "820          B69        S  \n",
       "823         E121        S  \n",
       "835          E49        C  \n",
       "839          C47        C  \n",
       "849          C92        C  \n",
       "853          D28        S  \n",
       "857          E17        S  \n",
       "862          D17        S  \n",
       "867          A24        S  \n",
       "871          D35        S  \n",
       "872  B51 B53 B55        S  \n",
       "879          C50        C  \n",
       "887          B42        S  \n",
       "889         C148        C  \n",
       "\n",
       "[202 rows x 10 columns]"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Agora vamos codificar label categóricas em labes numéricas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "le_sex = LabelEncoder()\n",
    "le_cabin = LabelEncoder()\n",
    "le_embarked = LabelEncoder()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LabelEncoder()"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "le_sex.fit(df.Sex)\n",
    "le_cabin.fit(df.Cabin)\n",
    "le_embarked.fit(df.Embarked)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"Sex\"] = le_sex.transform(df[\"Sex\"])\n",
    "df[\"Cabin\"] = le_cabin.transform(df[\"Cabin\"])\n",
    "df[\"Embarked\"] = le_embarked.transform(df[\"Embarked\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>80</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>54</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>54.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>51.8625</td>\n",
       "      <td>128</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>16.7000</td>\n",
       "      <td>144</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>58.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>26.5500</td>\n",
       "      <td>48</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    PassengerId  Survived  Pclass  Sex   Age  SibSp  Parch     Fare  Cabin  \\\n",
       "1             2         1       1    0  38.0      1      0  71.2833     80   \n",
       "3             4         1       1    0  35.0      1      0  53.1000     54   \n",
       "6             7         0       1    1  54.0      0      0  51.8625    128   \n",
       "10           11         1       3    0   4.0      1      1  16.7000    144   \n",
       "11           12         1       1    0  58.0      0      0  26.5500     48   \n",
       "\n",
       "    Embarked  \n",
       "1          0  \n",
       "3          2  \n",
       "6          2  \n",
       "10         2  \n",
       "11         2  "
      ]
     },
     "execution_count": 143,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Verificando a correlação dos dados"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PassengerId</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.111985</td>\n",
       "      <td>-0.084147</td>\n",
       "      <td>0.000877</td>\n",
       "      <td>0.028736</td>\n",
       "      <td>-0.081137</td>\n",
       "      <td>-0.064538</td>\n",
       "      <td>0.017465</td>\n",
       "      <td>-0.072897</td>\n",
       "      <td>0.031825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Survived</th>\n",
       "      <td>0.111985</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.030513</td>\n",
       "      <td>-0.545297</td>\n",
       "      <td>-0.231887</td>\n",
       "      <td>0.138202</td>\n",
       "      <td>0.042456</td>\n",
       "      <td>0.128261</td>\n",
       "      <td>0.038628</td>\n",
       "      <td>-0.130910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pclass</th>\n",
       "      <td>-0.084147</td>\n",
       "      <td>-0.030513</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.060014</td>\n",
       "      <td>-0.287184</td>\n",
       "      <td>-0.086972</td>\n",
       "      <td>0.056288</td>\n",
       "      <td>-0.311740</td>\n",
       "      <td>0.494000</td>\n",
       "      <td>0.170303</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sex</th>\n",
       "      <td>0.000877</td>\n",
       "      <td>-0.545297</td>\n",
       "      <td>-0.060014</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.167361</td>\n",
       "      <td>-0.152552</td>\n",
       "      <td>-0.110574</td>\n",
       "      <td>-0.137185</td>\n",
       "      <td>-0.083768</td>\n",
       "      <td>0.096805</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Age</th>\n",
       "      <td>0.028736</td>\n",
       "      <td>-0.231887</td>\n",
       "      <td>-0.287184</td>\n",
       "      <td>0.167361</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.139881</td>\n",
       "      <td>-0.246928</td>\n",
       "      <td>-0.076680</td>\n",
       "      <td>-0.125576</td>\n",
       "      <td>-0.090462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SibSp</th>\n",
       "      <td>-0.081137</td>\n",
       "      <td>0.138202</td>\n",
       "      <td>-0.086972</td>\n",
       "      <td>-0.152552</td>\n",
       "      <td>-0.139881</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.262348</td>\n",
       "      <td>0.291777</td>\n",
       "      <td>0.056745</td>\n",
       "      <td>0.002228</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Parch</th>\n",
       "      <td>-0.064538</td>\n",
       "      <td>0.042456</td>\n",
       "      <td>0.056288</td>\n",
       "      <td>-0.110574</td>\n",
       "      <td>-0.246928</td>\n",
       "      <td>0.262348</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.384970</td>\n",
       "      <td>0.001291</td>\n",
       "      <td>0.061455</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fare</th>\n",
       "      <td>0.017465</td>\n",
       "      <td>0.128261</td>\n",
       "      <td>-0.311740</td>\n",
       "      <td>-0.137185</td>\n",
       "      <td>-0.076680</td>\n",
       "      <td>0.291777</td>\n",
       "      <td>0.384970</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.262818</td>\n",
       "      <td>-0.239213</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cabin</th>\n",
       "      <td>-0.072897</td>\n",
       "      <td>0.038628</td>\n",
       "      <td>0.494000</td>\n",
       "      <td>-0.083768</td>\n",
       "      <td>-0.125576</td>\n",
       "      <td>0.056745</td>\n",
       "      <td>0.001291</td>\n",
       "      <td>-0.262818</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.231418</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Embarked</th>\n",
       "      <td>0.031825</td>\n",
       "      <td>-0.130910</td>\n",
       "      <td>0.170303</td>\n",
       "      <td>0.096805</td>\n",
       "      <td>-0.090462</td>\n",
       "      <td>0.002228</td>\n",
       "      <td>0.061455</td>\n",
       "      <td>-0.239213</td>\n",
       "      <td>0.231418</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             PassengerId  Survived    Pclass       Sex       Age     SibSp  \\\n",
       "PassengerId     1.000000  0.111985 -0.084147  0.000877  0.028736 -0.081137   \n",
       "Survived        0.111985  1.000000 -0.030513 -0.545297 -0.231887  0.138202   \n",
       "Pclass         -0.084147 -0.030513  1.000000 -0.060014 -0.287184 -0.086972   \n",
       "Sex             0.000877 -0.545297 -0.060014  1.000000  0.167361 -0.152552   \n",
       "Age             0.028736 -0.231887 -0.287184  0.167361  1.000000 -0.139881   \n",
       "SibSp          -0.081137  0.138202 -0.086972 -0.152552 -0.139881  1.000000   \n",
       "Parch          -0.064538  0.042456  0.056288 -0.110574 -0.246928  0.262348   \n",
       "Fare            0.017465  0.128261 -0.311740 -0.137185 -0.076680  0.291777   \n",
       "Cabin          -0.072897  0.038628  0.494000 -0.083768 -0.125576  0.056745   \n",
       "Embarked        0.031825 -0.130910  0.170303  0.096805 -0.090462  0.002228   \n",
       "\n",
       "                Parch      Fare     Cabin  Embarked  \n",
       "PassengerId -0.064538  0.017465 -0.072897  0.031825  \n",
       "Survived     0.042456  0.128261  0.038628 -0.130910  \n",
       "Pclass       0.056288 -0.311740  0.494000  0.170303  \n",
       "Sex         -0.110574 -0.137185 -0.083768  0.096805  \n",
       "Age         -0.246928 -0.076680 -0.125576 -0.090462  \n",
       "SibSp        0.262348  0.291777  0.056745  0.002228  \n",
       "Parch        1.000000  0.384970  0.001291  0.061455  \n",
       "Fare         0.384970  1.000000 -0.262818 -0.239213  \n",
       "Cabin        0.001291 -0.262818  1.000000  0.231418  \n",
       "Embarked     0.061455 -0.239213  0.231418  1.000000  "
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.corr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Normalizando as features numericas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = MinMaxScaler().fit_transform(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.        ,  1.        ,  0.        , ...,  0.13913574,\n",
       "         0.55172414,  0.        ],\n",
       "       [ 0.00225225,  1.        ,  0.        , ...,  0.1036443 ,\n",
       "         0.37241379,  1.        ],\n",
       "       [ 0.00563063,  0.        ,  0.        , ...,  0.10122886,\n",
       "         0.88275862,  1.        ],\n",
       "       ..., \n",
       "       [ 0.98873874,  1.        ,  0.        , ...,  0.16231419,\n",
       "         0.47586207,  0.        ],\n",
       "       [ 0.99774775,  1.        ,  0.        , ...,  0.0585561 ,\n",
       "         0.2       ,  1.        ],\n",
       "       [ 1.        ,  1.        ,  0.        , ...,  0.0585561 ,\n",
       "         0.40689655,  0.        ]])"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Dividindo em treino e teste"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df[:,:-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(202, 9)"
      ]
     },
     "execution_count": 152,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = df[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(202,)"
      ]
     },
     "execution_count": 155,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
